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DETAILED ACTION 

1 . In view of the appeal brief filed on August 1 9, 2009, PROSECUTION IS HEREBY 
REOPENED. New grounds of rejections are set forth below. 

To avoid abandonment of the application, appellant must exercise one of the 
following two options: 

(1 ) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply 
under 37 CFR 1.113 (if this Office action is final); or, 

(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41 .31 followed 
by an appeal brief under 37 CFR 41 .37. The previously paid notice of appeal fee and 
appeal brief fee can be applied to the new appeal. If, however, the appeal fees set forth 
in 37 CFR 41 .20 have been increased since they were previously paid, then appellant 
must pay the difference between the increased fees and the amount previously paid. 

A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by 
signing below: 

Response to Arguments 

2. Applicant's arguments, see pages 13-15, filed August 19, 2009, with respect to 
rejected claims 51-53 under 35 U.S.C. 103(a) have been fully considered and are 
persuasive. The rejections of claims 51-53 have been withdrawn. 
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3. Applicant's arguments filed August 19, 2009 with respect to claims 1-5, 7, 10-25, 
28-38, 40, 42, 44-47, 54, and 56 have been fully considered but they are not 
persuasive. 

4. Applicants argues the following: 

a. Appellant submits that Takahashi merely teaches that the user can go 
back and enter the degree-of-importance/scene length for a particular tag, after 
the video has been captured. (See, e.g., para. 00120. "As described above, since 
auxiliary information is generated in connection with the shooting operation, the 
user can easily generate auxiliary information at shooting. Further, PointOfView 
may be inputted after shooting.") Thus, while Takahashi teaches generating 
some auxiliary information during video capture - specifically, the Viewpoint tag 
within the PointofView descriptor as shown in FIG. 3 - other auxiliary information 
- specifically, the Value for the PointOfView descriptor in FIG. 3, which describes 
degree-of-importance - is entered by the user after video capture. Thus, since 
Takahashi does not teach automatically determining scene-length for-a-person- 
of-interest, the Examiner's reliance on this feature in Takahashi as teaching the 
claimed "saliency circuitry for automatically generating. ..second saliency signal" 
is misplaced. 

> The Examiner disagrees. As discussed in Tf 001 9 of Takahashi, the level 
of importance of a particular scene would have a value between 0 (0 = not 
important) and 1 (1 = most important) with increments of 0.1 that can be 
determined based on the length of a scene where the son and the daughter 
appear, the excitement of the user by using a pressure sensor of a sweat sensor 
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or by having a measurement of loudness in the scene being captured (See H 
01 19). Takahashi further discloses that the level of importance which can be set 
manually could also be set automatically by use of said pressure, sweat or 
loudness sensors (Also in U 01 19). This clearly teaches automatically generating 
an image related saliency signal in response to the image signal as claimed. 

b. Thus, since Takahashi does not teach automatically determining scene- 
length for-a-person-of-interest, the Examiner's reliance on this feature in 
Takahashi as teaching the claimed "saliency circuitry for automatically 
generating... second saliency signal" is misplaced. 

The Office Action also contends that in the alternative, a different teaching 
in Takahashi discloses the claimed "saliency circuitry for automatically 
generating.., second saliency signal", as follows: 

The Examiner disagrees with the Applicant. However, that Examiner 
would like to point out that Takahashi teaches that the saliency signal can 
be either input by the user as well as automatically generated by the 
camera upon detection of excitement of the user using either a pressure 
sensor or a sweat sensor provided in the camera so that the camera 
would automatically assign a "degree of importance" based on the 
detected excitement of the user as also discussed in 1 1 01 19 (Office 
Action, p. 4, emphasis added.) 

Thus, this portion of the Office Action specifically contends that 
Takahashl's "exciting" value corresponds to the automatically generated "second 
saliency signal" recited in claim 44. Appellant disagrees, and submits that the 
Examiner's reading of Takahashi goes beyond the explicit and even implicit 
teachings of the reference. 
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As noted above, para. 0019 of Taka hash i teaches tags such as "son", 
"daughter" and "exciting", and degree-of-importance for each tag. With regard to 
the "exciting" tag relied upon in the rejection, Takahashi further discloses that in 
one embodiment the "exciting" tag can have a binary degree-of-importance. 
Takahashi then contrasts this binary-valued embodiment with various other 
embodiments which allow for additional levels of "exciting", such as one in which 
the user selects levels of "exciting" in 0.1 increments by pressing a series of 
buttons, and another in which a user directly inputs a level for "exciting" via an 
"exciting button". Finally, Takahashi another embodiment in which "a value of 
exciting may be inputted by sensing the degree of exciting of the user from the 
fingers of the user which are put on a pressure sensor or a sweat sensor 
provided at the upper surface of the body". Thus, Appellant submits that 
Takahashi does not teach that the pressure sensor or sweat sensor generates a 
saliency signal, but instead that the sensor generates a degree-of-importance 
value for the "exciting" tag. 

> As discussed above, by teaching that a level of importance can be given 
to the persons in the image (regardless of whether they are identified as a tag or 
not), the level of importance is still being generated upon the detection of the 
excitement of the user which is detected by the pressure sensor of the sweat 
sensor, which based on the measurement of the sweat or pressure would 
generated a value for the level of interest (which the Examiner is interpreting as 
the second saliency signal as claimed, noting that the saliency signal is also a 
level of importance given to a particular scene). 
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c. Takahashi fails to teach, disclose or suggest at least "circuitry for 

combining said saliency signals to provide a complex saliency signal" 

The Office Action contends that Takahashi teaches this as follows: 

Takahashi discloses that the saliency signals (wherein the Examiner is 
interpreting the person information and the degree of importance as saliency 
signals) are displayed together (See figs. 20(a) and 26(b)). Therefore, by at least 
teaching that the person information and the degree of importance are displayed 
together in the display of the camera, Takahashi teaches combining the signals 
to create a complex signal that is used for creating a display signal to inform the 
user about the person information and the degree of importance in the video 
being displayed. Therefore, the Examiner understands that Takahashi discloses 
"circuitry for combining said saliency signals to provide a complex saliency 
signal" as claimed. (Office Action, p. 5, emphasis added.) 

Appellant respectfully disagrees with the contention. Appellant assumes 

(for the sake of argument) that the person-of-interest information and the degree- 

of-importance information are each a "saliency signal". Even so, Takahashi does 

not teach or suggest "combining" these two pieces of information. Takahashi 

merely teaches simultaneous display of these two separate pieces of information 

(see, e.g., Figs. 26(a) and (b)), which Appellant respectfully submits is not the 

same as "combining" as recited in claim 44. Furthermore, even assuming (for the 

sake of argument) that simultaneous display of these separate two pieces of 

information is the same as combining them, Appellant respectfully that the 

displayed information is not "complex" as recited in claim 44. 

> It is noted that the word "complex" can be defined as something that is 

composed of two or more units. Furthermore, although the specifications recite 

"The different individual saliency signals could be combined in real time, or used 

or recorded separately. Combination of saliency signals could be effected 
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algebraically or logically to give a composite saliency signal; effective 
combination could also be effected by ascertaining (for example via a look-up 
table) into what portion of a multidimensional saliency space the saliency signals 
are jointly located, and developing a composite saliency signaf, the claim does 
not require any particular method to have the saliency signals combined. 
Although the claims are interpreted in light of the specification, limitations from 
the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Therefore, by teaching that the 
persons information and the degree of importance information is stored/displayed 
together in association with the image, Takahashi discloses circuitry for 
combining said saliency signals to form a complex saliency signal as claimed 
since the person information, the level of importance and the image are 
combined in the displayed together (or combined). 

d. Claims 1 and 21 recite "a circuit for providing a maximum value for the 

saliency signal contemporaneously with activation of the picture taking control". 

Appellant respectfully submits that the proposed combination fails to teach, 

disclose, or suggest at least this feature. The Office Action relies on Metcalfe for 

teaching this feature, as follows: 

The Examiner noted that Metcalfe further discloses that the record button 
1 1 2 and the button 1 1 1 can be combined into a single combination key, 
and depressing the combination key starts the recording of video while 
subsequent pressure applied by the user generates a LOI signal for 
recording with the captured video frames (See page 6, lines 6-1 2) ....by 
teaching that the combination button is initially depress for activation while 
a subsequent pressure applied by the user that in response the camera 
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would assign a particular LOI to the video frames being recorded, Metcalfe 
discloses "a circuit for providing a maximum value for the saliency signal 
contemporaneously with activation of the picture taking button" as claimed 
since even if the LOI is not assign at the same time, the L01 is assign at a 
time that can be considered to be occurring during a particular period of 
time. Therefore, the Examiner understands that the combined teaching of 
Metcalfe in view of Takahashi as discussed and analyzed in claims 1 and 
21 teaches all the limitations as presented. (Office Action, pp. 20-21 .) 

Appellant disagrees with this contention. Appellant assumes (for the sake 

of argument) that the level of interest (LOI) signal in Metcalfe corresponds to a 

saliency signal, but the value provided is then a current value of the LOI, not "a 

maximum value" as recited in claims 1 and 21 . 

Nor does Takahashi disclose this feature. Even assuming (for the sake of 
argument) that the auxiliary information in Takahashi corresponds to a saliency 
signal, Takahashi does not disclose "providing a maximum value" for the 
auxiliary information. 

Accordingly, Appellant submits that the relied-upon references do not 
(individually or in combination) disclose, teach, or suggest the claimed feature 
described above. Therefore, a prima facie case of obviousness for claims 1 and 
21 has not been made, and the rejection should be withdrawn. 
> The Examiner disagrees. As discussed in the previous Office Action, 
Metcalfe discloses that the level of interest that can be set to the images can 
have values that can be variably assigned through the capture of the video 
sequence. This teaches the use of a circuit that provides a maximum value for 
the saliency signal contemporaneously with activation of the picture taking 
control since by teaching that the value can be varied, Metcalfe inherently 
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discloses that the circuit can provide a maximum value as well as a minimum 
value as desired by the user with respect to the scene being taken. Metcalfe 
further discloses that the record button 112 and the button 1 1 1 can be combined 
into a single combination key, and depressing the combination key starts the 
recording of video while subsequent pressure applied by the user generates a 
LOI signal for recording with the captured video frames (See page 6, lines 6-12). 
It is noted that the word "contemporaneous" is defined as "Originating, existing, 
or happening during the same period of time" (Definition from "The American 
Heritage® Dictionary of the English Language", Fourth Edition, 2000). Therefore, 
by teaching that the combination button is initially depress for activation while a 
subsequent pressure applied by the user that in response the camera would 
assign a particular LOI to the video frames being recorded, Metcalfe discloses "a 
circuit for providing a maximum value for the saliency signal contemporaneously 
with activation of the picture taking button" as claimed since even if the LOI is not 
assign at the same time, the LOI is assign at a time that can be considered to be 
occurring during a particular period of time). 

e. Appellant respectfully submits that claim 40 is allowable for at least the 
reason that the proposed combination of Metcalfe in view of Takahashi does not 
disclose, teach, or suggest at least the feature of "saliency circuitry for combining 
said first and second saliency signals to form a complex saliency signal". The 
Office Action acknowledges (p. 35) that Metcalfe does not explicitly disclose this 
feature, but contends that Takahashi teaches the feature as follows: 
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saliency circuitry for combining said saliency signals to form a complex saliency 
signal (the Examiner is reading the complex saliency signal as the combined 
information having the persons information and the degree of importance 
information as shown in Takahashi... (As shown in figs. 23, the user would set 
the auxiliary information and the set auxiliary information would be displayed in 
the display as shown in figs. 25 and 26 (see persons information and the degree 
of importance displayed on the display which the examiner is reading as the 
complex saliency signal generated) (Office Action, p. 37, emphasis added.) 

Appellant respectfully disagrees with the contention. Appellant assumes 
(for the sake of argument) that the person-of-interest information and the degree- 
of-importance information are each a "saliency signal". Even so, Takahashi does 
not teach or suggest "combining" these two pieces of information. Takahashi 
merely teaches simultaneous display of these two separate pieces of information 
(see, e.g., Figs. 26(a) and (b)), which Appellant respectfully submits is not the 
same as "combining" as recited in claim 40. Furthermore, even assuming (for the 
sake of argument) that simultaneous display of these separate two pieces of 
information is the same as combining them, Appellant respectfully that the 
displayed information is not "complex" as recited in claim 40. 

Accordingly, Appellant submits that the relied-upon references do not 
(individually or in combination) disclose, teach, or suggest the claimed feature 
described above. Therefore, a prima facie case of obviousness for claim 40 has 
not been made, and the rejection should be withdrawn. 
> The Examiner disagrees. It is noted that the word "complex" can be 
defined as something that is composed of two or more units. Furthermore, 
although the specifications recite "The different individual saliency signals could 
be combined in real time, or used or recorded separately. Combination of 
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saliency signals could be effected algebraically or logically to give a composite 
saliency signal; effective combination could also be effected by ascertaining (for 
example via a look-up table) into what portion of a multidimensional saliency 
space the saliency signals are jointly located, and developing a composite 
saliency signaf, the claim does not require any particular method to have the 
saliency signals combined. Although the claims are interpreted in light of the 
specification, limitations from the specification are not read into the claims. See 
In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Therefore, 
by teaching that the persons information and the degree of importance 
information is stored/displayed together in association with the image, Takahashi 
discloses circuitry for combining said saliency signals to form a complex saliency 
signal as claimed since the person information, the level of importance and the 
image are combined in the displayed together (or combined). 

f. Appellant respectfully submits that Metcalfe in view of Takahashi fails to 
teach, disclose, or suggest "a user operable control for generating a non- 
playback saliency signal". The Office Action alleges that Metcalfe teaches this 
feature as follows: 

Metcalfe discloses.., a user operable control (button 12 as shown in fig. 1) for 
generating a non-playback saliency signal generated in Metcalfe is a level of 
interest signal to indicate portion of the image signals that have certain degree of 
interest to be stored in the memory 120 in association with the saliency signal) 
(Office Action, p. 41 .) 

Appellant respectfully disagrees with the contention. Appellant assumes 



(for the sake of argument) that the level of interest (LOI) signal is a "saliency 
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signal". However, in further discussing the LOI, the Office Action notes that 
"when reproducing, the camera would select particular images based on the 
degree of importance (LOI) as set by the user when recording the images (See 
pages 5-7; page 6, line 31 - page 7, line 8.)" (Emphasis added.) Since the LOI 
signal is used during reproduction, Appellant submits that the level of interest 
signal in Metcalfe is a playback signal. In contrast, claim 54 recites a "non- 
playback saliency signal". 

The Office Action also contends (p. 43) that the auxiliary information in 
Takahashi is a saliency signal. However, para. 0127 of Takahashi discloses that 
the auxiliary information is used during playback, where claim 54 requires a "non- 
playback saliency signal". 

Accordingly, Appellant submits that the relied-upon references do not 
(individually or in combination) disclose, teach, or suggest the claimed feature 
described above. Therefore, a prima facie case of obviousness for claim 54 has 
not been made, and the rejection should be withdrawn 
> The Examiner disagrees. As discussed in the previous Office Action by 
teaching that the images would be reproduced or played-back based on the level 
of interest in which. Thus, if the image has a low level of importance, the image 
would be skipped and the images with high level of importance would be 
reproduced. By teaching that only images with high level of importance are to be 
displayed, Metcalfe discloses the use of both non-playback signal (for images 
with low level of interest) and play-back signal (for images with high level of 
interest) (See pages 5-6). Therefore, Metcalfe discloses a user operable control 
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for generating a non-playback signal as claimed since the level of interest is 
determined by the user operation or excitement as discussed in Metcalfe. 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

6. Claims 44-47 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Takahashi, US 2002/0041757 A1. 

7. Regarding claim 44, Takahashi discloses an imaging system comprising an 
electronic camera (See figs. 4(a), 4(b), 8, 9(a), 9(b), 1 0, and 1 1 ) for producing an image 
signal, a physically or mechanically operable user control (Takahashi discloses the use 
of buttons 109 to set auxiliary information (which the Examiner is interpreting as the 
saliency information) related to the image data being captured; page 6, If 0106. 
Takahashi further discloses the use of a pressure sensor 109a and sweat sensor 109b 
to determine the auxiliary information related to the image data (page 6, If 01 1 2 - page 
7, U 01 14) for receiving an input from a user and for generating a first saliency signal 
while the image signal is being produced (Takahashi further discloses that the auxiliary 
information includes information related to the persons (i.e. son, daughter, friend, father, 
mother) (see figs. 23(a) and 23 (b); page 8, If 01 1 7 - page 9, If 01 24)) for receiving an 
input from a user and for generating, in response to the input from the user a saliency 



Application/Control Number: 10/696,567 Page 14 

Art Unit: 2622 

signal (auxiliary information having information such as persons information)), saliency 
circuitry for automatically generating an image related second saliency signal in 
response to the image signal (Takahashi further teaches that the auxiliary information 
includes a degree of importance of said persons appearing in the image data, and that 
said degree of importance can be determined based on the time length of a scene 
where a particular person set by the user appear; see page 8, If 01 19. Takahashi 
further teaches that the level of importance of a particular scene would have a value 
between 0 (0 = not important) and 1 (1 = most important) with increments of 0.1 that can 
be determined based on the excitement of the user by using a pressure sensor of a 
sweat sensor or by having a measurement of loudness in the scene being captured 
(See U 01 19). Takahashi further discloses that the level of importance which can be set 
manually could also be set automatically by use of said pressure, sweat or loudness 
sensors (Also in U 01 19)), and circuitry for combining said saliency signals to form a 
complex saliency signal (Takahashi discloses that the auxiliary information has persons 
information and degree of importance of said persons and that the auxiliary information 
is stored with the image signal and further discloses that the camera would also display 
the auxiliary information on a display screen (As shown in figs. 23, the user would set 
the auxiliary information and the set auxiliary information would be displayed in the 
display as shown in figs. 25 and 26 (see persons information and the degree of 
importance displayed on the display); page 8, If 01 1 8 - page 9, U 01 24)) (This teaches 
the combination of the two saliency signals (the persons information and the degree of 
importance information stored/displayed together in association with the image) to 
create a complex saliency signal as claimed. It is noted that the word "complex" can be 
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defined as a combination or a plurality of parts or elements. Therefore, by teaching that 
the person's information and the degree of importance information is stored/displayed 
together in association with the image, Takahashi discloses circuitry for combining said 
saliency signals to form a complex saliency signal as claimed). 

8. Regarding claim 45, Takahashi discloses that the operation of at least a part of 
the electronic camera is arranged to be controlled in response to the complex saliency 
signal (Takahashi discloses that the auxiliary information has persons information and 
degree of importance of said persons and that the auxiliary information is stored with 
the image signal and further discloses that the camera would also display the auxiliary 
information on a display screen (As shown in figs. 23, the user would set the auxiliary 
information and the set auxiliary information would be displayed in the display as shown 
in figs. 25 and 26 (see persons information and the degree of importance displayed on 
the display); page 8, H 01 18 - page 9, If 0124)) being arranged to be controlled in 
response to the saliency signal (based on the auxiliary information the display would 
display the auxiliary information as shown in figs. 25 and 26). 

9. Regarding claim 46, Takahashi discloses a separate user operable picture 
taking control for permitting enabling the electronic camera to take pictures (shooting 
button 104 as shown in fig. 4 (b)). 
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1 0. Regarding claim 47, Takahashi discloses that the first saliency signal can have 
more than two values (As shown in figs. 23 (a) and 23 (b), Takahashi discloses that 
more than one person can be selected (i.e. the son and the daughter); page 8, If 01 19). 

Claim Rejections - 35 USC § 103 

1 1 . The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

12. Claims 1, 3-5, 10-21, 23-25, 28-38, 40, 42, 54 and 56 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Metcalfe, AU 743216 B in view of 
Takahashi, US 2002/0041757 A1. 

13. Regarding claim 1, claim 1 recites "... a saliency signal that (a) can change in 
value between at least three different discrete values while the image signal is being 
produced, or (b) can have values that are continuously variable while the image signal 
is being produced". It is noted by the Examiner that the term " can " is non-limiting and 
therefore has not been given patentable weight during examination of the claims on 
their merits. Language that suggests or makes optional but does not require steps to be 
performed or does not limit a claim to a particular structure does not limit the scope of a 
claim or claim limitation. MPEP §2106. 

The subject matter of a properly construed claim is defined by the terms that limit its 
scope. It is this subject matter that must be examined. As a general matter, the 
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grammar and intended meaning of terms used in a claim will dictate whether the 
language limits the claim scope. Language that suggests or makes optional but does 
not require steps to be performed or does not limit a claim to a particular structure does 
not limit the scope of a claim or claim limitation. The following are examples of language 
that may raise a question as to the limiting effect of the language in a claim: 

(A) statements of intended use or field of use, 

(B) "adapted to" or "adapted for" clauses, 

(C) "wherein" clauses, or 

(D) "whereby" clauses. 

This list of examples is not intended to be exhaustive. See also MPEP § 21 1 1 .04. 
USPTO personnel are to give claims their broadest reasonable interpretation in light of 
the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 1023, 
1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited in the 
claim should not be read into the claim. E-Pass Techs., Inc. v. 3Com Corp., 343 F.3d 
1364, 1369, 67 USPQ2d 1947, 1950 (Fed. Cir. 2003) (claims must be interpreted "in 
view of the specification" without importing limitations from the specification into the 
claims unnecessarily). In re Prater, 415 F.2d 1393, 1404-05, 162 USPQ 541, 550- 551 
(CCPA 1969). See also In re Zletz, 893 F.2d 319, 321-22, 13 USPQ2d 1320, 1322 
(Fed. Cir. 1989) ("During patent examination the pending claims must be interpreted as 
broadly as their terms reasonably allow.... The reason is simply that during patent 
prosecution when claims can be amended, ambiguities should be recognized, scope 
and breadth of language explored, and clarification imposed.... An essential purpose of 
patent examination is to fashion claims that are precise, clear, correct, and 
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unambiguous. Only in this way can uncertainties of claim scope be removed, as much 
as possible, during the administrative process."). 

Metcalfe discloses a camera apparatus (See fig. 1) comprising an electronic 
camera (See fig. 1) for producing an image signal, a user operable picture taking control 
(111 as shown in fig. 1 ) for selectively activating the electronic camera to take pictures, 
and an additional physically or mechanically operable user control (1 12 as shown in fig. 
1 , note that the limitations "an additional physically or mechanically operable user 
control" are written as optional elements by using the word "or") for receiving an input 
from a user and for generating, in response to the input from the user a saliency signal 
(Metcalfe discloses generating a level of interest (LOI) set by the user when using 
button 112) that (a) can change in value between at least three different discrete values 
while the image signal is being produced (Metcalfe discloses assigning a level of 
interest to the image data being recorded, wherein the user can variably assign a 
plurality of level of interest through the capture of the a video sequence; see Fig. 3. 
This teaches generating a saliency signal that can change in value between at least 
three different discrete values while the image signal is being produced), or (b) can 
have values that are continuously variable while the image signal is being produced 
(Metcalfe discloses assigning a level of interest to the image data being recorded, 
wherein the user can variably assign a plurality of level of interest through the capture of 
the a video sequence; see Fig. 3. This teaches generating a saliency signal that can 
have values that are continuously variable while the image signal is being produced), a 
circuit for providing a maximum value for the saliency signal contemporaneously with 
activation of the picture taking button (By teaching that the level of interest that can be 
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set to the images can have values that can be variably assigned through the capture of 
the video sequence, Metcalfe discloses the use of a circuit that provides a maximum 
value for the saliency signal contemporaneously with activation of the picture taking 
control since the circuit in Metcalfe can provide a maximum value as well as a minimum 
value as desired by the user with respect to the scene being taken) (Metcalfe further 
discloses that the record button 112 and the button 111 can be combined into a single 
combination key, and depressing the combination key starts the recording of video while 
subsequent pressure applied by the user generates a LOI signal for recording with the 
captured video frames (See page 6, lines 6-12). It is noted that the word 
"contemporaneous" is defined as "Originating, existing, or happening during the same 
period of time" (Definition from "The American Heritage® Dictionary of the English 
Language", Fourth Edition, 2000). Therefore, by teaching that the combination button is 
initially depress for activation while a subsequent pressure applied by the user that in 
response the camera would assign a particular LOI to the video frames being recorded, 
Metcalfe discloses "a circuit for providing a maximum value for the saliency signal 
contemporaneously with activation of the picture taking button" as claimed since even if 
the LOI is not assign at the same time, the LOI is assign at a time that can be 
considered to be occurring during a particular period of time), and a memory (120 as 
shown in fig. 1) arranged for storing the image signal and the saliency signal (page 4, 
lines 5-13; page 5, lines 4-35), operation of at least a part of the camera apparatus 
while the electronic camera is activated to take pictures (Metcalfe discloses setting the 
LOI while the camera is activated to take pictures, and storing said LOI associated with 
the images so that said LOI can be used to control the reproduction of the images (i.e. 
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printing, creating thumbnail files for photo albums, etc.)) being arranged to be controlled 
in response to the saliency signal (as discussed in page 6, line 31 - page 7, line 8 , 
Metcalfe discloses that the saliency signal (LOI) can be used to control the reproduction 
of the images (i.e. printing, creating thumbnail files for photo albums, etc.)) (See page 4, 
line 5 - page 5, line 23; page 6, line 23 - page 7, line 8). 

Metcalfe does not explicitly disclose that the operation in response to the 
saliency signal is in addition to recording the saliency signal in the memory. 

However, Takahashi discloses an electronic camera (See figs. 4(a), 4(b), 8, 9(a), 
9(b), 10, and 11) producing image signal, comprising a user operable picture taking 
control (shooting button 104 as shown in fig. 4 (b)) for selectively activating the 
electronic camera to take pictures, and an additional operable user control (Takahashi 
discloses the use of buttons 109 to set auxiliary information (which the Examiner is 
interpreting as the saliency information) related to the image data being captured; page 
6, U 01 06. Takahashi further discloses the use of a pressure sensor 1 09a and sweat 
sensor 109b to determine the auxiliary information related to the image data (page 6, If 
01 12 - page 7, If 01 14). Also Takahashi discloses that the auxiliary information 
includes information related to the persons (i.e. son, daughter, friend, father, mother) 
and a degree of importance of said persons appearing in the image data (see figs. 23(a) 
and 23 (b); page 8, If 01 17 - page 9, If 0124)) for receiving an input from a user and for 
generating, in response to the input from the user a saliency signal (auxiliary information 
having information such as persons information and degree of importance of the 
persons in the image) that (a) can change in value between at least three different 
discrete values while the image signal is being produced (as shown in figs. 23(a) and 
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23(b), the persons information and degree of importance of the persons in the image 
can change in value between a plurality of values (i.e. three or more values as 
claimed)), or (b) can have values that are continuously variable while the image signal is 
being produced (the auxiliary information can be changed during the capture of the 
image signal), and a memory (13) arranged for storing the image signal and the 
saliency signal (Takahashi further teaches storing the auxiliary information in the header 
of the scene; page 8, If 01 23 - page 9, 01 26), operation of at least a part of the 
camera apparatus while the electronic camera is activated to take pictures (Takahashi 
discloses recording the auxiliary information and further discloses that the camera 
would also display the auxiliary information on a display screen (As shown in figs. 23, 
the user would set the auxiliary information and the set auxiliary information would be 
displayed in the display as shown in figs. 25 and 26; page 8, H 01 18 - page 9, U 0124)) 
being arranged to be controlled in response to the saliency signal (based on the 
auxiliary information the display would display the auxiliary information as shown in figs. 
25 and 26), the operation in response to the saliency signal is in addition to recording 
the saliency signal in the memory (The Examiner is interpreting displaying the auxiliary 
information using the display of the camera as the operation that is in addition to 
recording the saliency signal in the memory as claimed). Displaying the auxiliary 
information while capturing the image data is advantageous because it would provide 
the user with a user friendly interface that would allow changing the degree of 
importance of the images being captured and would also allow the user to be aware of 
the information being added to the image data. 
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Therefore, taking the combined teaching of Metcalfe in view of Takahashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the concept of having the camera further displaying the 
salient signal being assign to the image data as taught in Takahashi to modify the 
teaching of Metcalfe to perform an operation in response to the saliency signal in 
addition to recording the saliency signal in the memory. The motivation to do so would 
have been to provide the user with a user friendly interface that would allow changing 
the degree of importance of the images being captured and would also allow the user to 
be aware of the information being added to the image data. 

14. Regarding claim 3, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches that wherein said part includes a 
buffer for receiving said image signal, the buffer having a capacity arranged to be 
controlled by the value of the saliency signal during operation of the camera apparatus 
(In a further embodiment, Takahashi discloses the concept of determining the amount of 
data to be transmitted based on the cost of transmission service; wherein the allowed 
length of data (L) is determined and compared to the amount of data to be transmitted, 
if the amount of data to be transmitted the apparatus would select video section with a 
priority higher than a threshold value. If after selecting those video with the priority 
higher than a threshold, the length of data still more than the allowed length, the 
apparatus would increase the priority threshold, however, if the length is less that the 
allowed length, the video signals are transmitted (Page 12, ^ 0151-0154). By teaching 
adjusting the amount of data to be transmitted based on the length allowed for 
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transmission and also based on the priority of the video signals to be transmitted, 
Takahashi inherently discloses "a buffer for receiving said image signal, the buffer 
having a capacity arranged to be controlled by the value of the saliency signal during 
operation of the camera apparatus" as claimed since the amount being adjusted in the 
apparatus needs to be in a particular buffer or memory prior to transmission in order to 
properly select the video signals with higher priority). One of an ordinary skill in the art 
would have found obvious to apply the concepts of adjusting the amount of data to be 
transmitted based on the priority set to the video signals to further modify the camera 
apparatus to have a buffer for receiving said image signal, the buffer having a capacity 
arranged to be controlled by the value of the saliency signal during operation of the 
camera apparatus with the motivation of select as many as possible the most important 
video signals from the video data for transmission as suggested by Takahashi (Page 
12,110154). 

1 5. Regarding claim 4, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches that the part includes image selection 
circuitry for receiving the saliency and image signals and for selectively passing ones of 
said image signals as determined by said saliency signal (As taught in Metcalfe, the 
camera receives the saliency signal (LOI) and based on said saliency signal, when 
reproducing, the camera would select particular images based on the degree of 
importance as set by the user when recording the images. By teaching that the images 
with a high LOI would be selected for creating an album or to be printed with high 
quality further teaches that images that do not have a high LOI would be passed or 
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skipped since only the images with a high LOI would be displayed in the virtual album. 
Metcalfe as applied reads on "... said part includes image selection circuitry for 
receiving the saliency and image signals and for selectively passing ones of said image 
signals as determined by said saliency signaf as written since the display or skip of 
images is determined based on the LOI set. (See pages 5-7, specifically page 6, line 31 
- page 7, line 8)). 

16. Regarding claim 5, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches that said part comprises the memory 
(memory 13 in Takahashi), the memory including management circuitry arranged to be 
responsive to the saliency signal for selectively retaining in said memory images 
associated with higher saliency levels in preference to images with lower saliency levels 
(Takahashi further discloses that the images are recorded or transmitted based on the 
importance level of the image, wherein only images with high importance level can be 
recorded in order to reduce the amount of use of the recording medium; page 5, H 
0094). 

17. Regarding claim 10, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches a user operable control for picture 
taking control of the electronic camera (Metcalfe, 1 1 1 as shown in fig. 1). 

18. Regarding claim 1 1 , Metcalfe discloses that the user control includes a normal 
picture taking control on the electronic camera (111 as shown in fig. 1). 
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1 9. Regarding claim 12, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claim 1 further teaches at least one further physically or 
mechanically operable user control for generating a corresponding related saliency 
signal (As shown in Takahashi, figs. 4(b): 109; fig. 9(a): 109, 10: 109a, and 11: 109b, 
Takahashi discloses the use of a plurality of buttons to select from different auxiliary 
information to be assigned to the video signal (note that the buttons 109 are physically 
located on the camera). Furthermore, as shown in figs 23(a) and 23(b), Takahashi 
further discloses that the auxiliary information can be selected using a monitor, wherein 
the user can select the person information and the degree of importance of the persons 
using buttons 103m, 103n, 101m and 101n; page 8, U 0119 (although these buttons are 
displayed on the touch screen display, the Examiner understands that the buttons are 
physically located on the camera since they have different physical location on said 
display)). Grounds for rejecting claim 1 apply here. 

20. Regarding claim 13, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claims 1 and 12 further teaches saliency circuitry for 
combining said saliency signals to form a complex saliency signal (the Examiner is 
reading the complex saliency signal as the combined information having the persons 
information and the degree of importance information as shown in Takahashi), the 
complex saliency signal being the saliency signal for controlling at least a part (The 
Examiner is reading the part as the display of the camera in Takahashi as discussed in 
claim 1) of the electronic camera and the saliency signal the memory is arranged to 
store (As discussed in claim 1, Takahashi discloses that the auxiliary information has 
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persons information and degree of importance of said persons and that the auxiliary 
information is stored with the image signal and further discloses that the camera would 
also display the auxiliary information on a display screen (As shown in figs. 23, the user 
would set the auxiliary information and the set auxiliary information would be displayed 
in the display as shown in figs. 25 and 26 (see persons information and the degree of 
importance displayed on the display); page 8, 1f 01 1 8 - page 9, U 01 24)) being arranged 
to be controlled in response to the saliency signal (based on the auxiliary information 
the display would display the auxiliary information as shown in figs. 25 and 26). 
Grounds for rejecting claim 1 and 12 apply here. 

21 . Regarding claim 14, Metcalfe discloses a saliency circuitry for generating an 
image related saliency signal in response to said image signal (Metcalfe discloses 
controlling the reproduction operation of the camera based on the associated saliency 
signal to the image signal so that when reproducing the image signal with higher 
importance would be displayed differently from the other image signal. See page 4, line 
5 - page 5, line 23; page 6, line 23 - page 7, line 8. Takahashi further discloses 
reproducing the video based on the importance level of the image, wherein only images 
with high importance level can be reproduced so that the user can enjoy the recorded 
work without feeling tired, and the power consumption is reduced to secure more driving 
time; page 10, If 01 31). 

22. Regarding claim 15, limitations have been discussed and analyzed in claim 13. 
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23. Regarding claim 16, Metcalfe discloses circuitry for incorporating said saliency 
signal in each of said image signals (Metcalfe discloses incorporating said saliency 
signal (LOI) to each of the frames in the image signal. See page 4, line 5 - page 5, line 
23; page 6, line 23 - page 7, line 8). 

24. Regarding claim 17, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claims 1 teaches that the user control is part of the camera 
or is physically attached to the camera body (See Metcalfe, controls in Fig. 1 ; see also 
Takahashi buttons 109 as shown in figs. 4(b), 9(b), 10 and 11, and buttons 103m, 103n, 
101m, 101 n as shown in figs. 23(a) and 23(b)). 

25. Regarding claim 18, he combined teaching of Metcalfe in view of Takahashi 
fails to teach that the user control is a remote control for communication with the 
camera. However, the Examiner takes Official Notice that the concept of controlling a 
camera with a remote control is well known in the art at the time the invention was made 
and that it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to have made the user control of Metcalfe and Takahashi a remote 
control as opposed to a camera-body integrated control. One would have been 
motivated to do so because it is well known in the art that by using a remote control to 
control some elements of a camera, the user does not have to be near the camera to 
send and receive desired signals from the camera. This is particularly advantageous in 
cases where plural cameras are used or cameras are placed out of the reach of the 
user (e.g. surveillance cameras), where the remote control would allow the user to send 
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signals to the camera(s) from a separate location, thereby simplifying camera control for 
the user. 

26. Regarding claim 19, Metcalfe discloses that the user control comprises a 
physically movable control member (pressure button that assign a level of interest of an 
image signal based on the pressure applied to said button) and a sensor arranged to be 
responsive to movement of the control member (See page 4, line 27 - page 5, line 4). 
By teaching a pressure button that assigns a LOI of an image signal based on the 
pressure applied to the button, Metcalfe discloses a physically movable control member 
(in this case the pressure button as discussed in Metcalfe) and a sensor arranged to be 
responsive to movement of the control member since by teaching that the LOI is 
assigned based on the pressure applied to the button, the use of a sensor to determine 
the applied pressure in order to assigned the LOI of an image is inherent and 
necessitated in Metcalfe. 

27. Regarding claim 20, the Examiner notes that the limitations "the user control 
comprises a pressure or force sensing transducer for deriving the saliency signal that 
can have values that are continuously variable", the elements "pressure" or "force 
sensing transducer" are written as optional elements by using the word "or". 

Metcalfe discloses that the user control comprises a pressure or force sensing 
transducer for deriving the saliency signal that can have values that are continuously 
variable (Metcalfe as applied to claim 19, teaches the use of pressure (pressure button 
that assign a level of interest of an image signal based on the pressure applied to said 



Application/Control Number: 10/696,567 Page 29 

Art Unit: 2622 

button; page 4, line 27 - page 5, line 4) for deriving the saliency signal. By teaching 
that Metcalfe discloses the use of pressure applied to the button, Metcalfe discloses the 
use of pressure for deriving the saliency signal as claimed. 

28. Regarding claim 21 , limitations have been discussed and analyzed in claim 1 . 

29. Regarding claim 23, limitations have been discussed and analyzed in claim 4. 

30. Regarding claim 24, limitations have been discussed and analyzed in claim 3. 

31 . Regarding claim 25, limitations have been discussed and analyzed in claim 5. 

32. Regarding claim 28, limitations have been discussed and analyzed in claim 10. 

33. Regarding claim 29, limitations have been discussed and analyzed in claim 1 1 . 

34. Regarding claim 30, limitations have been discussed and analyzed in claim 12. 

35. Regarding claim 31, limitations have been discussed and analyzed in claim 13. 

36. Regarding claim 32, limitations have been discussed and analyzed in claim 13. 

37. Regarding claim 33, limitations have been discussed and analyzed in claim 13. 
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38. Regarding claim 34, limitations have been discussed and analyzed in claim 16. 

39. Regarding claim 35, limitations have been discussed and analyzed in claim 17. 

40. Regarding claim 36, limitations have been discussed and analyzed in claim 18. 

41 . Regarding claim 37, limitations have been discussed and analyzed in claim 19. 

42. Regarding claim 38, limitations have been discussed and analyzed in claims 19 
and 20. 

43. Regarding claim 40, claim 40 recites "... one of the saliency signals being a 
signal that (a) can change in value between at least three different discrete values while 
the image signal is being produced, or (b) can have values that are continuously 
variable while the image signal is being produced". It is noted by the Examiner that the 
term " can " is non-limiting and therefore has not been given patentable weight during 
examination of the claims on their merits. Language that suggests or makes optional 
but does not require steps to be performed or does not limit a claim to a particular 
structure does not limit the scope of a claim or claim limitation. MPEP §21 06. 

The subject matter of a properly construed claim is defined by the terms that limit its 
scope. It is this subject matter that must be examined. As a general matter, the 
grammar and intended meaning of terms used in a claim will dictate whether the 
language limits the claim scope. Language that suggests or makes optional but does 
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not require steps to be performed or does not limit a claim to a particular structure does 
not limit the scope of a claim or claim limitation. The following are examples of language 
that may raise a question as to the limiting effect of the language in a claim: 

(A) statements of intended use or field of use, 

(B) "adapted to" or "adapted for" clauses, 

(C) "wherein" clauses, or 

(D) "whereby" clauses. 

This list of examples is not intended to be exhaustive. See also MPEP § 21 1 1 .04. 
USPTO personnel are to give claims their broadest reasonable interpretation in light of 
the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 1023, 
1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited in the 
claim should not be read into the claim. E-Pass Techs., Inc. v. 3Com Corp., 343 F.3d 
1364, 1369, 67 USPQ2d 1947, 1950 (Fed. Cir. 2003) (claims must be interpreted "in 
view of the specification" without importing limitations from the specification into the 
claims unnecessarily). In re Prater, 415 F.2d 1393, 1404-05, 162 USPQ 541, 550- 551 
(CCPA 1969). See also In re Zletz, 893 F.2d 319, 321-22, 13 USPQ2d 1320, 1322 
(Fed. Cir. 1989) ("During patent examination the pending claims must be interpreted as 
broadly as their terms reasonably allow.... The reason is simply that during patent 
prosecution when claims can be amended, ambiguities should be recognized, scope 
and breadth of language explored, and clarification imposed.... An essential purpose of 
patent examination is to fashion claims that are precise, clear, correct, and 
unambiguous. Only in this way can uncertainties of claim scope be removed, as much 
as possible, during the administrative process."). 
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Metcalfe discloses an imaging system (See fig. 1) comprising an electronic 
camera (See fig. 1) for producing an image signal, physically or mechanically operable 
user controls (See user controls 106, 104, 111, 112, 110 and 108 as shown in fig. 1), 
the user control being arranged for receiving an input from a user and for generating 
first saliency signal (Metcalfe discloses the use of button 1 12 to generate a plurality of 
saliency signals (Level of interest signals "LOI") to be associated to the image signal 
being recorded with the camera) while the image signal is being produced, and saliency 
circuitry (the camera in Metcalfe inherently has a saliency signal circuitry to generate 
the saliency signal upon operation of the camera button 1 1 2) for storing said first 
saliency signal (Metcalfe discloses recording the plurality of saliency signals in a 
memory (tape 120 in fig. 1) in association with the image data; see page 4, line 5 - 
page 5, line 23; page 6, line 23 - page 7, line 8), the saliency signal being, a signal that 
(a) can change in value between at least three different discrete values while the image 
signal is being produced (Metcalfe discloses assigning a level of interest to the image 
data being recorded, wherein the user can variably assign a plurality of level of interest 
through the capture of the a video sequence; see Fig. 3. This teaches generating a 
saliency signal that can change in value between at least three different discrete values 
while the image signal is being produced), or (b) can have values that are continuously 
variable while the image signal is being produced (Metcalfe discloses assigning a level 
of interest to the image data being recorded, wherein the user can variably assign a 
plurality of level of interest through the capture of the a video sequence; see Fig. 3. 
This teaches generating a saliency signal that can have values that are continuously 
variable while the image signal is being produced), a memory (120 as shown in fig. 1) 
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arranged for storing the image signal and the saliency signal in response to the saliency 
signal (page 4, lines 5-1 3; page 5, lines 4-35), operation of at least part of the electronic 
camera being arranged to be controlled in response to the saliency signal (as discussed 
in page 6, line 31 - page 7, line 8, Metcalfe discloses that the saliency signal (LOI signal 
stored in memory 120) can be used to control the reproduction of the images (i.e. 
printing, creating thumbnail files for photo albums, etc.)) (See page 4, line 5 - page 5, 
line 23; page 6, line 23 - page 7, line 8). 

Metcalfe does not explicitly disclose at least two physically or mechanically 
operable user controls, each of the user controls for generating first and second 
saliency signals; that said saliency circuitry combines said first and second saliency 
signals to form a complex saliency signal; that the operation of at least part of the 
electronic camera being arranged to be controlled in response to the complex saliency 
signal. 

However, Takahashi discloses an electronic camera (See figs. 4(a), 4(b), 8, 9(a), 
9(b), 10, and 11) producing image signal, comprising a user operable picture taking 
control (shooting button 104) for selectively activating the electronic camera to take 
pictures, and at least two physically or mechanically operable user controls (As shown 
in Takahashi, figs. 4(b): 109; fig. 9(a): 109, 10: 109a, and 11: 109b, Takahashi discloses 
the use of a plurality of buttons to select from different auxiliary information (which the 
Examiner is interpreting as the saliency information) to be assigned to the video signal 
(note that the buttons 109 are physically located on the camera). Furthermore, as 
shown in figs 23(a) and 23(b), Takahashi further discloses that the auxiliary information 
can be selected using a monitor, wherein the user can select the person information 
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and the degree of importance of the persons using buttons 103m, 103n, 101m and 
101n; page 8, If 01 19 (although these buttons are displayed on the touch screen 
display, the Examiner understands that the buttons are physically located on the 
camera since they have different physical location on said display)). Takahashi further 
discloses the use of a pressure sensor 109a and sweat sensor 109b to determine the 
auxiliary information related to the image data (page 6, U 01 1 2 - page 7, ^ 011 4). Also 
Takahashi discloses that the auxiliary information includes information related to the 
persons (i.e. son, daughter, friend, father, mother) and a degree of importance of said 
persons appearing in the image data (see figs. 23(a) and 23 (b); page 8,^0117- page 
9, If 0124)) for receiving an input from a user and for generating, in response to the 
input from the user a saliency signal (auxiliary information having information such as 
persons information and degree of importance of the persons in the image) that (a) can 
change in value between at least three different discrete values while the image signal 
is being produced (as shown in figs. 23(a) and 23(b), the persons information and 
degree of importance of the persons in the image can change in value between a 
plurality of values (i.e. three or more values as claimed)), or (b) can have values that are 
continuously variable while the image signal is being produced (the auxiliary information 
can be changed during the capture of the image signal), and saliency circuitry for 
combining said saliency signals to form a complex saliency signal (Takahashi discloses 
that the auxiliary information has persons information and degree of importance of said 
persons and that the auxiliary information is stored with the image signal and further 
discloses that the camera would also display the auxiliary information on a display 
screen (As shown in figs. 23, the user would set the auxiliary information and the set 
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auxiliary information would be displayed in the display as shown in figs. 25 and 26 (see 
persons information and the degree of importance displayed on the display); page 8, If 
01 1 8 - page 9, H 01 24)) (This teaches the combination of the two saliency signals (the 
persons information and the degree of importance information stored/displayed together 
in association with the image) to create a complex saliency signal as claimed. It is 
noted that the word "complex" can be defined as a combination or a plurality of parts or 
elements. Therefore, by teaching that the person's information and the degree of 
importance information is stored/displayed together in association with the image, 
Takahashi discloses circuitry for combining said saliency signals to form a complex 
saliency signal as claimed), the complex saliency signal being the saliency signal for 
controlling at least a part (The Examiner is reading the part as the display of the camera 
in Takahashi as discussed in claim 1) of the electronic camera and the saliency signal 
the memory is arranged to store (As discussed in claim 1 , Takahashi discloses that the 
auxiliary information has persons information and degree of importance of said persons 
and that the auxiliary information is stored with the image signal and further discloses 
that the camera would also display the auxiliary information on a display screen (As 
shown in figs. 23, the user would set the auxiliary information and the set auxiliary 
information would be displayed in the display as shown in figs. 25 and 26 (see persons 
information and the degree of importance displayed on the display which the examiner 
is reading as the complex saliency signal generated) (This teaches performing an 
operation ); page 8, U 01 18 - page 9, If 0124)) being arranged to be controlled in 
response to the saliency signal (based on the auxiliary information the display would 
display the auxiliary information as shown in figs. 25 and 26). Therefore Takahashi 
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discloses at least two physically or mechanically operable user controls, each of the 
user controls for generating first and second saliency signals; that said saliency circuitry 
combines said first and second saliency signals to form a complex saliency signal; that 
the operation of at least part of the electronic camera being arranged to be controlled in 
response to the complex saliency signal as claimed. Having a plurality of operation 
controls to generate different saliency signals to be combined into a single complex 
saliency signal to control the display operation of the camera is advantageous because 
it would provide the user with a user friendly interface that would allow changing the 
degree of importance of the images being captured and would also allow the user to be 
aware of the information being added to the image data. 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the concept of having the camera with a plurality of 
operation controls to generate different saliency signals to be combined to control the 
operation of the display to further displaying the combined saliency signal being assign 
to the image data as taught in Takahashi to modify the teaching of Metcalfe to have at 
least two physically or mechanically operable user controls, each of the user controls for 
generating first and second saliency signals; that said saliency circuitry combines said 
first and second saliency signals to form a complex saliency signal; that the operation of 
at least part of the electronic camera being arranged to be controlled in response to the 
complex saliency signal. The motivation to do so would have been to provide the user 
with a user friendly interface that would allow changing the degree of importance of the 
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images being captured and would also allow the user to be aware of the information 
being added to the image data. 

44. Regarding claim 42, Metcalfe discloses a separate user operable picture taking 
control for selectively activating the electronic camera to take pictures (See button 1 1 1 
as shown in fig. 1). 

45. Regarding claim 54, claim 54 recites "an electronic camera having a picture 
taking control for selectively activating the camera to derive input picture signals, the 
electronic camera further including a user operable control for generating a saliency 
signal capable of having plural values and a buffer for receiving the input picture signals 
and having a capacity for the input picture signals determined in response to the value 
of the saliency signal". It is noted by the Examiner that the term " capable of . It is noted 
by the Examiner that the term "capable of is non-limiting and therefore has not been 
given patentable weight during examination of the claims on their merits. Language that 
suggests or makes optional but does not require steps to be performed or does not limit 
a claim to a particular structure does not limit the scope of a claim or claim limitation. 
MPEP §2106. 

The subject matter of a properly construed claim is defined by the terms that limit 
its scope. It is this subject matter that must be examined. As a general matter, the 
grammar and intended meaning of terms used in a claim will dictate whether the 
language limits the claim scope. Language that suggests or makes optional but does 
not require steps to be performed or does not limit a claim to a particular structure does 
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not limit the scope of a claim or claim limitation. The following are examples of language 
that may raise a question as to the limiting effect of the language in a claim: 

(A) statements of intended use or field of use, 

(B) "adapted to" or "adapted for" clauses, 

(C) "wherein" clauses, or 

(D) "whereby" clauses. 

This list of examples is not intended to be exhaustive. See also MPEP § 21 1 1 .04. 

USPTO personnel are to give claims their broadest reasonable interpretation in 
light of the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 
1023, 1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited 
in the claim should not be read into the claim. E-Pass Techs., Inc. v. 3Com Corp., 343 
F.3d 1364, 1369, 67 USPQ2d 1947, 1950 (Fed. Cir. 2003) (claims must be interpreted 
"in view of the specification" without importing limitations from the specification into the 
claims unnecessarily). In re Prater, 415 F.2d 1393, 1404-05, 162 USPQ 541, 550- 551 
(CCPA 1969). See also In re Zletz, 893 F.2d 319, 321-22, 13 USPQ2d 1320, 1322 
(Fed. Cir. 1989) ("During patent examination the pending claims must be interpreted as 
broadly as their terms reasonably allow.... The reason is simply that during patent 
prosecution when claims can be amended, ambiguities should be recognized, scope 
and breadth of language explored, and clarification imposed.... An essential purpose of 
patent examination is to fashion claims that are precise, clear, correct, and 
unambiguous. Only in this way can uncertainties of claim scope be removed, as much 
as possible, during the administrative process"). 
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Furthermore, it has been held that the recitation that an element is "capable of 
performing a function is not a positive limitation but only requires the ability to so 
perform. It does not constitute a limitation in any patentable sense. In re Hutchison, 69 
USPQ 138. 

Metcalfe discloses an apparatus (See fig. 1) comprising an electronic camera 
(See fig. 1 ) having a picture taking control (button 1 1 1 as shown in fig. 1 ) for selectively 
activating the camera to store an image to a memory (Fig. 1 : 120), the camera further 
including a user operable control (button 1 12 as shown in fig. 1 ) for generating a non- 
playback saliency signal (the saliency signal generated in Metcalfe is a level of interest 
signal to indicate portion of the image signals that have certain degree of interest to be 
stored in the memory 120 in association with the saliency signal) and picture selection 
circuitry (circuitry is inherent in the Metcalfe reference to control the recording and 
reproduction of the video signals stored in the memory 120) for selectively passing the 
picture signals in response to the saliency signal (Metcalfe discloses that the user can 
select a saliency signal (LOI) to a particular image being captured, the saliency signal is 
stored in association with the image (See pages 5-7). Metcalfe further discloses that 
based on the stored saliency signal, the images would be reproduced, wherein when 
reproducing, the camera would select particular images based on the degree of 
importance (LOI) as set by the user when recording the images (See pages 5-7; page 6, 
line 31 - page 7, line 8)). By teaching that the images with a high LOI would be 
selected for creating an album or to be printed with high quality further teaches that 
images that do not have a high LOI would be passed or skipped since only the images 
with a high LOI would be displayed in the virtual album. Metcalfe as applied reads on 
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the limitations "... picture selection circuitry for selectively passing the picture signals in 
response to the saliency signal ..." since the display or skip of images is determined 
based on the saliency signal (LOI)), the saliency signal being capable of having more 
than two values (As discussed by Metcalfe , the operation of the button 1 12 would 
automatically generate a plurality of saliency signals, wherein when operating the button 
112, if the button is set to be controlled by pressing it a plurality of times, it would 
represent creating a first plurality of saliency signals that would generate the plurality of 
saliency signals to be associated to the image signal when recording into the memory 
120) (See page 4, line 5 - page 5, line 23; page 6, line 23 - page 7, line 8). 

Although Metcalfe discloses selectively passing the image signals during 
reproduction of the video based on the saliency signal, Metcalfe does not explicitly 
disclose selectively passing the image to the memory in response to the saliency signal. 

However, Takahashi discloses an electronic camera (See figs. 4(a), 4(b), 8, 9(a), 
9(b), 10, and 11) producing image signal, comprising a user operable picture taking 
control (shooting button 104 as shown in fig. 4 (b)) for selectively activating the 
electronic camera to take pictures for further recording or transmission (Takahashi 
discloses that the captured image data would be either recorded or transmitted to 
another device (See page 5, U 0094-0097)), and an additional operable user control 
(Takahashi discloses the use of buttons 109 to set auxiliary information (which the 
Examiner is interpreting as the saliency information) related to the image data being 
captured; page 6, If 0106. Takahashi further discloses the use of a pressure sensor 
109a and sweat sensor 109b to determine the auxiliary information related to the image 
data (page 6, H 01 12 - page 7, ^ 0114). Also Takahashi discloses that the auxiliary 
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information includes information related to the persons (i.e. son, daughter, friend, father, 
mother) and a degree of importance of said persons appearing in the image data (see 
figs. 23(a) and 23 (b); page 8, If 01 17 - page 9, U 0124)) for receiving an input from a 
user and for generating, in response to the input from the user a saliency signal 
(auxiliary information having information such as persons information and degree of 
importance of the persons in the image) that (a) can change in value between at least 
three different discrete values while the image signal is being produced (as shown in 
figs. 23(a) and 23(b), the persons information and degree of importance of the persons 
in the image can change in value between a plurality of values (i.e. three or more values 
as claimed)), or (b) can have values that are continuously variable while the image 
signal is being produced (the auxiliary information can be changed during the capture of 
the image signal), operation of at least a part of the camera apparatus while the 
electronic camera is activated to take pictures (Takahashi discloses recording the 
auxiliary information and further discloses that the camera would also display the 
auxiliary information on a display screen (As shown in figs. 23, the user would set the 
auxiliary information and the set auxiliary information would be displayed in the display 
as shown in figs. 25 and 26; page 8,^0118- page 9, U 01 24)) being arranged to be 
controlled in response to the saliency signal (based on the auxiliary information the 
display would display the auxiliary information as shown in figs. 25 and 26). Also, 
Takahashi teaches that the important images from a captured video can be extracted so 
that only scenes of high degrees of importance can be efficiently recorded or 
transmitted as compared with the case where all of the shot scenes are stored or 
transmitted and that the extraction of the scenes of high degrees of importance may be 
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carried out during or after shooting of the scenes (See page 5, H 0094; page 10, H 0130) 
(This teaches the concept of selectively passing the image to the memory in response 
to the saliency signal as claimed since the image signals would be either recorded or 
transmitted based in the priority given to the scenes so that only important scenes are 
transmitted or recorded while said scenes are being captured). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the concept of selectively recording image data in a 
memory in response to an assigned priority to scenes in the image data as taught in 
Takahashi to modify the teaching of Metcalfe to selectively pass the image to the 
memory in response to the saliency signal. The motivation to do so would have been to 
reduce the amount of use in the recording medium and reduce the communication costs 
as suggested by Takahashi (Page 5, If 0094). 

46. Regarding claim 56, limitations have been discussed and analyzed in claim 54. 

47. Claims 2, 7, and 22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Metcalfe, AU 743216 B in view of Takahashi, US 2002/0041757 
A1 and further in view of Matsumoto et al., US Patent 6,795,642 B2. 

48. Regarding claim 2, although Takahashi teaches that a compression circuitry 
(See Takahashi, fig. 1: 15; page 5, H 0097; page 6, If 0105) for receiving the image 
signals and for compressing the image signals and that although the invention is 
described on the premise that a shot picture is recorded, a shot picture is not 
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necessarily recorded, and it can be used also when compressed video and audio data 
are transmitted as they are to be used on a network or the like (page 10, If 0130), the 
combined teaching of Metcalfe in view of Takahashi fails to teach compressing the 
image signals to an extent determined by the saliency signal. 

However, Matsumoto et al. teaches the concept of having a video recording 
apparatus (Fig. 2) of a surveillance system, recording video data captured by an 
electronic camera (See fig. 1), wherein when an alarm is activated, the importance of 
the video is determined to be high as compared to when the alarm is not activated (Col. 
3, lines 46-67). Matsumoto et al. further discloses that based on the degree of 
importance given to the video signal, the data compression is also adjusted (i.e. if the 
importance degree of the video is low, it would be compressed at high level and if the 
importance degree of the video is high, said video would be compressed at low level) 
(Col. 3, line 1 5 - col. 4, lines 1 9). Matsumoto also discloses that the importance level 
can also be adjusted by the user operating the surveillance system (Col. 7, lines 22-34). 
Matsumoto et al. further discloses that by adjusting the compression of the video being 
captured, it is possible to record the monitoring image data having a high degree of 
importance as much as possible (Col. 7, lines 35-43). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi and 
further in view of Matsumoto et al. as a whole, it would have been obvious to one of 
ordinary skill in the art a the time the invention was made to apply the concept of 
adjusting the compression of a video captured by an electronic camera based on a 
degree of importance assign to the video as discussed in Matsumoto et al. to modify the 
teaching of Metcalfe and Takahashi by compressing the image signals to an extent 
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determined by the saliency signal. The motivation to do so would have been to record 
the monitoring image data having a high degree of importance as much as possible as 
suggested in Matsumoto et al. 

49. Regarding claim 7, the combined teaching of Metcalfe in view of Takahashi as 
discussed and analyzed in claims 1 and 2 further teaches that said part comprises the 
memory (memory 13 in Takahashi), including management circuitry arranged to be 
responsive to the saliency signal for selectively retaining in said memory images 
associated with higher saliency levels in preference to images with lower saliency levels 
(Takahashi further discloses that the images are recorded or transmitted based on the 
importance level of the image, wherein only images with high importance level can be 
recorded in order to reduce the amount of use of the recording medium; page 5, If 
0094). 

50. Regarding claim 22, limitations have been discussed and analyzed in claim 2. 

51. Claims 51-53 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Misawa et al., US 2002/0118285 A1 in view of Hayashi, US 2002/0031349 A1. 

52. Regarding claim 51, claim 51 recites "an electronic camera having a picture 
taking control for selectively activating the camera to derive input picture signals, the 
electronic camera further including a user operable control for generating a saliency 
signal capable of having plural values and a buffer for receiving the input picture signals 
and having a capacity for the input picture signals determined in response to the value 
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of the saliency signal". It is noted by the Examiner that the term " capable of . It is noted 
by the Examiner that the term "capable of is non-limiting and therefore has not been 
given patentable weight during examination of the claims on their merits. Language that 
suggests or makes optional but does not require steps to be performed or does not limit 
a claim to a particular structure does not limit the scope of a claim or claim limitation. 
MPEP §2106. 

The subject matter of a properly construed claim is defined by the terms that limit 
its scope. It is this subject matter that must be examined. As a general matter, the 
grammar and intended meaning of terms used in a claim will dictate whether the 
language limits the claim scope. Language that suggests or makes optional but does 
not require steps to be performed or does not limit a claim to a particular structure does 
not limit the scope of a claim or claim limitation. The following are examples of language 
that may raise a question as to the limiting effect of the language in a claim: 

(A) statements of intended use or field of use, 

(B) "adapted to" or "adapted for" clauses, 

(C) "wherein" clauses, or 

(D) "whereby" clauses. 

This list of examples is not intended to be exhaustive. See also MPEP § 21 1 1 .04. 

USPTO personnel are to give claims their broadest reasonable interpretation in 
light of the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44 USPQ2d 
1023, 1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited 
in the claim should not be read into the claim. E-Pass Techs., Inc. v. 3Com Corp., 343 
F.3d 1364, 1369, 67 USPQ2d 1947, 1950 (Fed. Cir. 2003) (claims must be interpreted 
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"in view of the specification" without importing limitations from the specification into the 
claims unnecessarily). In re Prater, 415 F.2d 1393, 1404-05, 162 USPQ 541, 550- 551 
(CCPA 1969). See also In re Zletz, 893 F.2d 319, 321-22, 13 USPQ2d 1320, 1322 
(Fed. Cir. 1989) ("During patent examination the pending claims must be interpreted as 
broadly as their terms reasonably allow.... The reason is simply that during patent 
prosecution when claims can be amended, ambiguities should be recognized, scope 
and breadth of language explored, and clarification imposed.... An essential purpose of 
patent examination is to fashion claims that are precise, clear, correct, and 
unambiguous. Only in this way can uncertainties of claim scope be removed, as much 
as possible, during the administrative process.") 

Furthermore, it has been held that the recitation that an element is "capable of 
performing a function is not a positive limitation but only requires the ability to so 
perform. It does not constitute a limitation in any patentable sense. In re Hutchison, 69 
USPQ 138. 

Misawa et al. discloses an apparatus comprising an electronic camera (See figs. 
1 and 2) having a picture taking control for selectively activating the camera to derive 
input picture signal (shutter button 14 as shown in fig. 2), the electronic camera further 
including a user operable control (priority selection switch 16 as shown in fig. 2; page 2, 
H 0026-0029 and If 0035; page 3, If 0037-0044) for generating a saliency signal capable 
of having plural values (Note that the Examiner is interpreting the priority given to the 
images as the saliency signal as claimed) and a memory (Fig. 1 : 40) for receiving the 
input picture signals (Page 2, U 0033; page 3, U 0036-0040 and U 0043-0044) and 
having a capacity for the input picture signals determined in response to the value of the 
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saliency signal (Misawa et al. discloses that that when capturing images if the priority of 
the image being captured is high and the capacity of the memory 40 is not enough to 
store the image, images with lower priority would be erased to provide space to store 
the new image having higher capacity. In the case that the new captured image has a 
lower capacity compared to the images already stored in memory, said new image with 
lower capacity would not be stored and a message would be displayed indicating that 
no memory capacity is available (Page 3, 0038-0045). This teaches that the memory 
has a capacity for the input picture signals determined in response to the value of the 
saliency signal as claimed since the capacity of the memory is being controlled based 
on the priority given to the images). 

Misawa et al. does not explicitly disclose that said memory is a buffer memory. 

However, Hayashi discloses an electronic camera (See fig. 1) comprising a 
picture taking control for selectively activating the camera to derive input picture signal 
(Operating member 27 as shown in fig. 1 ; page 3, U 0050) and a buffer memory for 
receiving the input picture signals (Hayashi teaches that the microprocessor 25 includes 
a buffer that is capable of adjust its capacity in accordance with the requirements to 
meet a god display resolution and the minimum number of continuous shots to be 
guaranteed (Page 3, H 0054; page 4, ^ 0080)) and having a capacity for the input 
picture signals determined in response to the needs for properly store/display the image 
data (As discussed before, Hayashi discloses that the capacity of the buffer is modified 
to allow proper display and storage of the image data being captured so that the 
capacity of the buffer is either reduced or increased accordingly (Pages 4-5, H 0076- 
0026)). Hayashi further teaches that by adjusting the size or capacity of the buffer in 
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accordance to the data being captured, an improvement on the timing for processing 
image is attained so that shooting and recording time of images is reduced (Page 5, If 
0097-0098). 

Therefore, taking the combined teaching of Misawa et al. in view of Hayashi as a 
whole, it would have been obvious to one of an ordinary skill in the art at the time the 
invention was made to apply the teaching of using a buffer that has a capacity that can 
be adjusted as taught in Hayashi to modify said memory in Misawa et al. to use a buffer 
to store the image signals as an alternative to the memory 40. The motivation to do so 
would have been to improve the timing for processing image is attained so that shooting 
and recording time of images is reduced as suggested by Hayashi (Page 5, U 0097- 
0098). 

53. Regarding claim 52, Misawa et al. further teaches that the saliency signal has 
more than two values (Important, Normal and memo. See page 2, U 0027). 

54. Regarding claim 53, the combined teaching of Misawa et al. in view of Hayashi 
further teaches that the electronic camera includes the buffer (Misawa et al. discloses 
that the memory 40 is included in the camera and Hayashi further teaches the buffer 
being included in the electronic camera (Page 3, U 0054)). 

Conclusion 

55. Because new grounds of rejection have been presented to reject claims 51 -53, 
this Office Action is made NON-FINAL. 
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